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The  Deployment  of  Visual  Attention:  Final  Report  1999-2002 
F49620-0frl-0071 
Jeremy  Wolfe 


Status  of  Effort:  Abstract 

This  is  the  final  report  summarizing  research  on  the  deployment  of  attention  in  visual 
search.  In  visual  search,  observers  look  for  targets  among  distractor  items.  Models  of 
search  had  assumed  that  items  were  sampled  without  replacement.  Items,  rejected  as 
distractors,  would  not  be  revisited  during  the  search.  Research  reported  here  falsifies  that 
hypothesis.  Our  data  are  consistent  with  sampling  with  replacement  -  no  use  of 
information  about  rejected  distractors.  The  data  do  not  reject  models  that  posit  small 
amounts  of  memory  for  rejected  distractors  (e.g.  don’t  revisit  the  last  N  rejected  items;  N 
<  7).  Why  is  search  so  apparently  random?  Observers  could  have  the  benefits  of  memory 
for  rejected  distractors  if  they  searched  in  an  orderly  manner  (e.g.  “reading”  a  display 
from  left  to  right).  Our  second  line  of  experiments  shows  that  the  temporal  costs  of  such  a 
strategy  are  too  high.  Left  to  its  own  devices,  covert  visual  search  proceeds  at  about  20- 
30  items/second.  Commanded  shifts  of  attention  proceed  at  only  3-5  shifts/second.  For 
the  tasks  used  here,  anarchic  covert  search  will  be  faster  than  commanded,  orderly  search. 
Our  Guided  Search  model  is  adapted  readily  to  search  with  replacement.  Nature  may 
have  stumbled  on  a  way  to  perform  sophisticated  search  tasks  with  simple  tools. 
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Introduction 

This  report  discusses  the  work  accomplished  in  our  laboratory  during  the  period  1999- 
2002  with  the  assistance  of  funding  from  the  Air  Force  Office  of  Scientific  Research. 
There  were  three  main  areas  of  proposed  research  in  this  grant  period  and  they  will 
comprise  the  three  main  divisions  of  this  report.  They  are:  1)  the  role  of  memory  for 
rejected  distractors  in  visual  search,  2)  comparison  of  the  speed  of  volitional  and 
reflexive  deployments  of  attention,  and  3)  modeling  of  the  deployment  of  visual  search. 
This  report  will  be  fairly  brief  in  its  descriptions  of  our  findings.  The  more  detailed 
accounts  can  be  found  in  our  published  and  submitted  papers. 

Memory  in  visual  search 

In  visual  search  tasks,  observers  look  for  a  target  item  among  some  number  of  distractor 
items.  This  might  be  accomplished  by  deploying  attention  from  item  to  item  in  sequential 
manner  or  by  processing  all  items  in  parallel,  accumulating  evidence  that  specific  items 
were  distractors  or  targets.  For  many  years,  standard  “serial”  and  “parallel”  models  of 
visual  search  had  assumed  that  information  accumulated  steadily  during  the  course  of  a 
search.  This  was  particularly  explicit  in  the  case  of  serial,  self-terminating  models  (e.g. 
Feature  Integration  theory  and  the  first  versions  of  Guided  Search).  (Treisman  &  Gelade, 
1980;  Wolfe,  Cave,  &  Franzel,  1989).  Those  models  assumed  that  attention  was  deployed 
from  item  to  item,  sampling  the  display  without  replacement  until  the  target  was  found  or 
until  almost  all  candidate  targets  were  examined  and  rejected. 
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In  1998,  we  upset  this  particular  apple  cart  with  a  paper  in  Nature  with  the  provocative 
title  “Visual  search  has  no  memory”  (Horowitz  &  Wolfe,  1998).  The  experiment  was 
conceptually  simple.  We  had  observers  perform  a  typically  inefficient  search  for  a  “T” 
among  “L”s  .  We  compared  the  standard  static  case  to  a  dynamic  version  in  which  we 
replotted  all  items  in  new  locations  every  100  msec.  Our  reasoning  was  that  this  would 
thwart  most  strategies  for  accumulating  information  during  a  search.  Certainly,  it  would 
prevent  marking  or  inhibiting  rejected  distractors  in  a  sequential  manner.  Under  dynamic 
conditions,  search  should  proceed  with  replacement.  The  best  that  an  observer  could  do 
would  be  to  grab  an  item  or  two  at  random  on  each  frames.  If  standard  search  was  search 
without  replacement  and  dynamic  search  was  sampling  with  replacement,  straight¬ 
forward  probability  theory  says  that  the  slope  of  the  RT  x  set  size  functions,  the  standard 
measure  of  search  efficiency,  should  be  twice  as  great  in  the  dynamic  case  as  in  the  static. 
This  was  not  the  case.  In  several  versions  of  the  experiment,  search  efficiency  was  not 
significantly  different  in  the  two  cases  (though  overall  RTs  were  longer).  We  concluded 
that  the  search  process  in  the  two  conditions  were  similar.  If  dynamic  search  had  to  be 
search  with  replacement,  then  standard  static  search  might  also  be  search  with 
replacement.  Hence,  we  argued  that  visual  search  had  no  memory. 
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Figure  One:  In  dynamic  search,  the  items  are  replotted  in  random  locations 
repeatedly  during  the  course  of  a  search. 

This  claim  provoked  a  fair  amount  of  controversy.  In  some  cases,  this  was  the  result  of 
taking  the  title  of  the  paper  too  literally  and  too  sweepingly.  For  example,  we  never 
meant  to  suggest  that  observers  could  not  remember  when  they  found  a  target  (Gibson, 
Li,  Skow,  Brown,  &  Cooke,  2000).  However,  others  questioned  our  actual  claim,  that 
visual  search  was  search  with  replacement  (Hollingworth  &  Henderson,  2002; 
Kristjansson,  2000;  Melcher  &  Kowler,  2001;  Muller  &  von  Muhlenen,  2000;  Peterson, 
Kramer,  Wang,  Irwin,  &  McCarley,  2000;  Shore  &  Klein,  2000).  Consequently,  we  have 
conducted  several  different  series  of  experiments  in  order  to  assess  the  role  of  memory  in 
search.  To  summarize  our  current  thinking,  it  seems  clear  that  we  can  reject  any  model  of 
search  that  proposes  perfect  or  near  perfect  sampling  without  replacement.  We  cannot 
reject  pure  sampling  with  replacement.  However,  nor  can  we  reject  models  with  some 
memory  (e.g.  models  that  propose  that  observers  can  avoid  deploying  attention  to  the  last 
N  items  visited  where  N  is  fairly  small  number).  These  partial  memory  models  seem  the 
most  promising  at  the  present  time  (Arani,  Karwan,  &  Drury,  1984). 
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Dynamic  Search  Experiments 

One  problem  with  a  simple  dynamic  search  paradigm  is  that  one  would  not  need  to  move 
attention.  If  items  are  randomly  replotted,  one  could  “sit  and  wait”  for  the  target  to 
appear.  That  would  produce  data  that  resemble  sampling  with  replacement  (von 
Muhlenen,  Muller,  &  Muller,  2003).  We  have  conducted  several  versions  of  the 
experiment  in  an  effort  to  thwart  ‘sit  and  wait’  strategies.  We  have  restricted  targets  to 
specific  radial  distances  from  fixation.  The  distance  changes  from  trial  to  trial  and  the 
manipulation  is  not  known  to  the  observer.  An  observer  attending  at  or  near  fixation  (or, 
indeed,  on  any  single  location)  would  make  a  disastrous  number  of  errors.  Performance 
was  essentially  the  same  as  in  our  other  dynamic  search  tasks.  Similarly,  we  have 
restricted  targets  to  on  quadrant  of  the  field.  Von  Muhlenen  et  al  (2003)  have  shown  that 
a  sit  and  wait  strategy  can  mimic  our  results  if  it  is  assumed  that  observers  can  “sit”  on 
multiple  items.  However,  at  present  we  believe  that  the  bulk  of  the  data  suggest  that  our 
basic  result  is  not  an  artifact  of  such  strategies. 


Attention  “sits’  here 


Time 


Figure  Two:  In  a  ‘sit  and  wait”  strategy,  observers  would  only  monitor  a  few 
locations  in  the  display.  In  this  specific  instance,  that  would  work. 
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Ami  Kristjannson  found  different  results  when  he  used  larger  set  sizes  than  those  used  in 
our  original  paper  (Kristjansson,  2000).  Faced  with  this  issue,  we  ran  a  version  of  the 
basic  experiment  with  large  set  sizes.  We  also  slowed  the  frame  rate  to  2  Hz  in  order  to 
decrease  the  masking  effects  of  rapid  change.  These  slower  frame  rates  cannot  be  used 
with  small  set  sizes  because  search  with  or  without  replacement  would  be  finished  within 
a  single  frame  on  most  trials. 


Dynamic  fixed  locations 


Dynamic  random  locations 


Standard  Search 


memory  prediction 


Figure  Three:  RT  x  set  size  functions  for  standard  search  for  a  T  among  Ls  (solid 
circles)  and  for  two  versions  of  a  dynamic  search  task  (open).  If  standard  search  is 
search  without  replacement,  then  data  from  dynamic  search  (with  replacement) 
should  fall  on  the  dashed  line,  which  it  does  not. 


As  Figure  Three  makes  clear,  there  is  no  significant  difference  between  the  Dynamic  and 
Standard  conditions  of  this  experiment.  The  error  rates  were  comparable,  as  well,  in  this 
study.  The  two  dynamic  conditions  differ  in  the  placement  of  items.  In  the  “fixed 
location”  condition,  the  same  N  locations  were  used  on  all  frames.  In  the  “random 
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location”  condition,  items  could  be  plotted  in  any  screen  location  on  each  frame.  This  did 
not  matter  in  our  study.  Kristjansson  used  a  fixed  location  method  and,  at  a  faster  frame 
rate  than  we  used,  may  have  introduced  masking  effects  that  slowed  his  observers  in  the 
dynamic  condition. 

Memory  in  Search:  ART  methods 

The  dynamic  search  paradigm  has  the  advantage  of  being  immediately  comprehensible.  If 
you  replot  all  the  items  every  100  msec,  you  cannot  keep  track  of  rejected  distractors. 
However,  the  method  has  a  number  of  drawbacks.  RTs  are  typically  longer  in  dyanamic 
search.  Error  rates  are  higher.  These  facts  complicate  the  interpretation  of  the  results.  In 
our  view,  they  are  directly  related  to  the  degrading  of  the  stimulus  in  the  dynamic 
condition.  However,  whatever  their  cause,  the  problems  with  the  dynamic  search 
paradigm  make  it  important  to  develop  converging  evidence  from  other  methods.  That 
has  been  an  important  part  of  the  research  in  the  prior  grant  period. 

One  set  of  experiments  used  what  can  be  called  “attentional  reaction  time  (ART)” 
methods.  Our  goal  was  to  measure  the  time  at  which  a  target  item  was  found  more 
precisely  than  can  be  done  with  a  standard  RT  technique.  The  basic  method  is  illustrated 
in  Figure  Four 
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Figure  Four:  In  this  version  of  an  ART  task,  observers  could  be  asked  to  name  the 
color  of  a  target  letter  (e.g.  F).  If  the  named  color  was  the  Frame  1  color,  then  we 
know  that  the  target  was  found  before  the  transition  from  Frame  1  to  Frame  2. 

Observers  searched  for  a  specific  letter  (e.g.  the  “F”  in  Figure  Four).  The  letter  was 
always  present.  The  observer’s  task  was  to  name  the  color  of  the  letter.  At  some  time 
during  the  course  of  the  trial,  the  color  changed  (here  from  purple  to  blue).  Note  that  the 
display  is  otherwise  stable.  This  is  not  a  dynamic  search  experiment.  If  the  observer 
reported  that  the  “F”  had  been  purple,  then  we  know  that  the  item  was  found  before  the 
color  change.  By  varying  the  time  of  the  color  change,  we  estimate  the  probability  of 
finding  a  target  as  a  function  of  time.  That  estimate  can  be  corrected  for  guessing  by 
looking  at  the  errors  when  observers  name  a  color  that  was  neither  the  Frame  2  nor  the 
Frame  2  color  of  the  target  letter. 

This  method  makes  qualitatively  different  predictions  for  models  that  propose  sampling 
from  the  display  with  and  without  memory.  A  cartoon  is  shown  in  Figure  5. 
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Figure  Five:  Sampling  with  replacement  (amnesia)  predicts  a  exponential 
cumulative  distribution  function  (CDF)  for  the  proportion  of  targets  found  before  a 
color  change.  Sampling  without  replacement  predicts  a  linear  CDF. 


If  observers  sample  without  replacement,  then  the  probability  that  the  target  will  be  found 
is  a  linear  function  of  the  time.  For  instance,  if  there  were  ten  items  and  one  item  was 
processed  every  50  msec,  then  there  would  be  a  10%  chance  of  finding  the  target  in  the 
first  50  msec,  20%  in  100  msec  and  so  on.  If  observers  sample  with  replacement,  then  the 
probability  of  finding  the  target  in  any  epoch  (e.g.  50  msec)  is  fixed  (10%  in  this 
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example)  and  the  cumulative  probability  is  an  exponential  function  (0.1N,  where  N  is  the 
number  of  epochs). 


©  data 

.  amnesic  model 

-  memory  model 

—  limited-capacity  model 


Figure  Six:  Data  from  an  ART  experiment  showing  probability  of  correctly  naming 
the  initial  color  of  a  letter  as  a  function  of  the  time  prior  to  a  color  change. 

It  is  possible  to  fit  amnesic  (with  replacement)  and  memory  (without  replacement) 
models  to  the  data.  Fits  are  done  by  adjusting  the  rate  parameter  (how  many  letters  are 
processed  per  second).  Search  with  replacement  is  always  a  better  fit  to  the  data  than 
search  without. 

That  said,  it  is  probably  a  mistake  to  see  the  issue  of  memory  in  search  as  a  dichotomous 
choice  between  search  with  and  without  replacement.  There  are  several  families  of  partial 
models  to  be  considered.  For  example,  suppose  observers  can  keep  track  of  the  last  N 
items  and  can  avoid  revisiting  them?  Alternatively,  suppose  that  observers  can  keep  track 


Some  typical  data  are  shown  in  Figure  Six. 
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of  an  average  of  N  items  but  these  are  not  necessarily  the  last  N  items?  The  results  of  our 
experiments  rule  out  models  that  assume  perfect  sampling  without  replacement. 
However,  the  results  are  consistent  with  a  modest  memory  for  rejected  distractors.  In  the 
ART  experiments,  N-back  models  provide  good  fits  to  the  data  for  N  <  6. 

Multiple  target  experiments 

In  a  third  line  of  experiments,  we  asked  observers  to  count  the  number  of  targets  in  a 
display.  Specifically,  on  each  trial,  we  asked  if  there  were  at  least  N  targets  in  the  display. 
We  varied  the  actual  number  of  targets  in  the  display.  The  virtue  of  this  paradigm  is  that, 
like  the  ART  paradigm,  it  produces  qualitatively  different  predictions  for  search  with  and 
without  replacement.  First,  it  is  important  to  note  that  this  paradigm  is  based  on  the 
assumption  that  observers  do  remember  targets  once  they  have  been  found  (Gibson  et  al., 
2000).  That  being  the  case,  consider  the  situation  in  an  amnesic  search,  a  search  with 
replacement.  Each  time  a  target  is  found  there  are  fewer  targets  to  find.  If  observers 
sample  at  random,  then  the  ratio  of  targets  to  distractors  gets  worse  over  time.  If  there 
were  five  targets  and  ten  distractors,  for  example,  the  observer  would  have  a  5/15=l/3rd 
chance  of  finding  a  target  at  the  start  of  the  trial  but  only  a  1/1 1  chance  by  the  time  four 
targets  had  been  found.  Therefore,  RT  should  be  an  accelerating  function  of  the  number 
of  targets  that  need  to  be  found. 

On  the  other  hand,  if  observers  sample  without  replacement,  then  the  number  of 
remaining  distractors  should  decline  in  step  with  the  number  of  remaining  targets.  The 
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theoretical  result  is  a  roughly  linear  increase  in  RT  with  number  of  targets  to  find.  This  is 
shown  in  Figure  7a. 


targets  to  look  for  targets  to  look  for 

Figure  Seven:  Hypothetical  (left)  and  actual  (right)  RTs  for  an  experiment  in  which 


observers  respond  positively  if  they  find  N  targets.  On  left,  solid  symbols  are  for  a 
model  that  samples  with  replacement  (amnesia).  Open  symbols  represent  sampling 
without  replacement  (memory) 


Figure  7b  shows  some  of  the  data.  The  number  above  each  curve  indicates  the  number  of 
target  present  in  the  display.  The  graphs  plot  RT  as  a  function  of  the  number  of  targets 
that  the  observer  was  search  for.  It  is  clear  that  the  data  in  7b  show  the  accelerating 
curvature  predicted  by  the  amnesic  /  with  replacment  model  and,  indeed,  the  model  fits 
are  stronger  for  with  replacement  than  for  without  replacement  models.  As  with  our  other 
experiments  in  this  area,  the  data  can  be  used  to  reject  a  model  that  argues  for  perfect 
memory  for  rejected  distractors.  The  data  are  consistent  with  a  no  memory  account. 
However,  the  data  cannot  reject  the  possibility  of  some  small  amount  of  memory. 
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This  represents  the  current  state  of  our  thinking  in  this  area.  We  cannot  find  convincing 
empirical  support  for  the  presence  of  memory  in  covert  visual  search.  At  the  much  slower 
time  course  of  overt  deployments  of  the  eyes,  there  is  some  evidence  for  memory  from 
other  labs  (e.g.  Hollingworth  &  Henderson,  2002;  Melcher  &  Kowler,  2001;  Peterson  et 
al.,  2000).  Because  of  the  range  of  possible  models  (Arani  et  al.,  1984)  (Horowitz  & 
Wolfe,  2003),  it  has  proven  difficult  to  date  to  design  experiments  that  can  clearly 
distinguish  between  no  memory  for  rejected  distractors  and  a  little  memory.  This  is  an 
area  of  ongoing  work. 

The  Costs  of  Attentional  Strategy 

Why  not  make  use  of  memory  for  rejected  distractors  in  search?  A  change  in  one  line  of 
code  is  all  that  it  takes  in  computer  models  of  search.  Given  that  memory  would  produce 
a  two-fold  improvement  in  search  efficiency,  why  not  use  it?  A  reasonable  guess  would 
be  that  there  is  some  cost  to  keeping  track  of  where  you  have  been  that  more  than 
eliminates  the  benefits  of  doing  so.  Direct  evidence  for  the  cost  of  holding  distractors  in 
memory  is  not  presently  available.  However,  we  have  shown  a  substantial  cost  for  a 
“short-cut’  that  would  mimic  memory.  If  observers  searched  a  display  in  an  orderly 
manner  (left  to  right,  center  to  periphery),  this  would  mimic  memory.  If  you  are 
searching  for  a  word  on  a  page  by  reading  from  left  to  right,  for  instance,  you  will  find 
the  target  word,  if  present,  on  average  after  reading  half  the  page.  In  this  case,  you  only 
need  to  remember  where  you  are  currently  and  the  rule  for  getting  to  the  next  place. 
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We  asked  about  the  time  course  of  such  volitional  deployments  of  attention.  In  a  series 
of  experiments,  we  forced  subjects  to  move  their  attention  in  a  specific  manner.  Figure 
Eight  shows  a  schematic  of  such  a  task. 


metacontrast  mask  is  followed  by  a  53  msec  letter  display  and  a  post  mask 

In  this  task,  observers  are  asked  to  move  their  attention  around  a  ring  of  items  at  a  fixed 
rate.  The  task  is  to  report  the  color  of  the  letter  “Y”.  The  items  change  from  frame  to 
frame  with  the  “Y”  appearing  only  on  frame  N.  On  frame  N,  the  “Y”  appears  in  position 
N.  The  observer  starts  at  position  zero.  Suppose  that  the  target  appears  in  position  8  at 
frame  8.  After  8  frames,  the  observer  should  have  deployed  attention  to  position  8  and, 
thus,  be  able  to  perform  the  task.  If  the  deployments  were  not  keeping  pace  with  the 
frame  rate,  the  observer  would  need  to  guess.  We  varied  the  frame  rate  using  a  staircase 
procedure  to  estimate  a  67%  threshold  duration  for  this  “Commanded”  search.  The  whole 
display  lasted  for  12  frames.  We  compared  this  to  a  control  condition  with  targets  present 
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on  all  12  frames.  This  permitted  free  “anarchic”  visual  search  and  allowed  us  to  estimate 
the  speed  of  deployment  when  attention  was  not  constrained  to  move  from  location  to 
location  in  a  fixed  order.  The  Commanded  search  threshold  averaged  274  msec  per 
deployment.  The  Anarchic  threshold  was  only  85  msec/item  which  is  very  close  to  the 
floor  value  of  80  msec  in  this  particular  design. 

In  order  to  eliminate  the  need  to  switch  deployment  rate  from  trial  to  trial,  we  repeated 
the  experiment  with  frame  duration  fixed  a  point  2/3"1  of  the  temporal  distance  between 
the  shorter  Anarchic  staircase  threshold  and  the  longer  Commanded  condition.  Accuracy 
in  the  Anarchic  condition  was  near  ceiling  (97%).  Accuracy  in  the  Anarchic  condition 
was  much  worse  (66%). 

We  have  performed  several  variants  and  control  experiments  using  this  basic  paradigm 
and  we  find  that  estimates  of  the  speed  of  Commanded  deployments  of  attentional  are 
always  markedly  slower  than  Anarchic  deployments.  Details  can  be  found  in  Wolfe, 
Alvarez,  and  Horowitz  (2000)  and  our  submitted  manuscript  on  this  topic  (Horowitz, 
Wolfe,  and  Alvarez,  submitted). 

This  method  has  some  of  the  limitations  of  the  dynamic  search  tasks.  Stimuli  are 
flickering  on  and  off.  There  are  masks.  While  we  have  attempted  to  control  for  factors 
other  than  visual  attention  deployment  speed,  the  situation  is  complicated.  Accordingly 
we  have  tried  to  gain  converging  evidence  from  another  method.  The  alternate  method  is 
shown  in  Figure  9,  below. 
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Figure  Nine:  Observers  were  instructed  to  report  the  identity  of  the  first  mirror 
reversed  letter  starting  at  12  o’clock  and  moving  clockwise.  Position  around  the 
circle  is  analogous  to  set  size  in  this  paradigm. 


In  this  task,  observers  were  asked  to  find  the  first  mirror-reversed  letter  and  to  identify  it 
as  an  S  or  a  P.  “First”  was  defined  by  asking  observers  to  begin  at  12  o’clock  at  the  top  of 
the  circle  and  find  the  first  target  as  they  proceeded  clockwise  around  the  circle.  Since 
there  can  be  multiple  mirror-reversed  items  on  each  trial,  this  strongly  encourages 
observers  to  move  attention  in  a  fixed  path  around  the  circle.  Target  distance  around  the 
circle  serves  the  same  role  as  set  size  in  a  standard  search  task.  In  Figure  9a,  the  observer 
makes  2  deployments  before  finding  the  target.  In  9b,  the  observer  makes  nine 
deployment.  (RT9-RT2)/7  gives  an  estimate  of  rate  of  deployment. 

In  the  actual  experiment,  set  sizes  of  7,  9,  and  11  were  used.  The  rate  of  deployment  is 
estimated  by  the  slope  of  the  function  relating  RT  to  target  position  around  the  circle  of 
items.  Note  that,  in  this  condition,  RT  should  be  dependent  only  on  serial  position  around 
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the  circle,  not  on  set  size.  A  target  in  position  5  should  be  found  after  five  deployments 
regardless  of  the  number  of  items  in  the  display  as  a  whole. 

For  comparison,  we  ran  an  Anarchic  condition  with  the  same  set  sizes.  In  this  case,  there 
was  only  on  mirror-reversed  letter  per  display  and  observers  were  simply  told  to  find  it. 
They  were  explicitly  told  that  they  did  not  need  to  search  in  any  specific  order.  In  this 
condition,  RT  should  not  be  dependent  on  position  around  the  circle.  However,  it  should 
be  dependent  on  set  size  in  the  usual  manner  of  visual  search  experiments. 

The  data  a  shown  in  Figure  Ten.  Fig.  10a  shows  the  results  for  the  Anarchic  condition. 
Note  that  the  RTs  do  not  depend  on  target  position.  However,  as  would  be  expected,  the 
RTs  for  smaller  set  sizes  are  faster  than  those  for  larger  set  sizes.  RT  x  set  size  slopes 
were  computed  from  target  positions  1-7  since  all  set  sizes  shared  those  positions.  The 
slope  was  58  msec/item.  This  yields  an  estimated  rate  of  deployment  between  58  and  1 16 
msec/item  (depending  on  models  of  memory  for  rejected  distractors.  We  would  lean 
toward  the  faster  —  minimal  memory  —  estimate,  on  the  basis  of  experiments  described 
above). 
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Figure  Ten:  Average  RTs  for  two  varieties  of  a  search  for  a  mirror  reversed  letter. 
In  10a,  observers  search  in  the  usual  manner  for  a  single  item.  In  10b,  observers 
search  for  the  first  position  (clockwise  from  12  o’clock)  that  contains  a  mirror- 
reversed  letter.  The  slope  of  the  RT  x  target  position  function  estimates  the  speed  of 
volitionally  commanded  deployments  of  attention. 


Turning  to  the  Command  condition  data  in  10b,  we  see  that  RT  is  linearly  dependent  on 
target  position.  Set  size  does  not  have  an  effect.  The  slope  of  this  RT  x  position  function 
is  196  msec/item,  significantly  slower  than  even  the  most  conservative  estimate  from  the 
Anarchic  condition. 

To  summarize,  in  a  range  of  experiments,  we  have  estimated  the  rate  at  which  attention 
can  be  deployed  in  an  orderly,  commanded  fashion.  Regardless  of  our  method,  the 
answer  is  always  about  the  same.  Volitional  deployments  of  attention  take  200-300  msec 
per  deployment.  Estimates  of  deployment  rates  in  standard  visual  search  are  always  much 
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faster,  perhaps  20-60  msec/item.  Even  if  we  assumed  a  classic  model  of  sampling  without 
replacement,  volitional  deployments  would  be  slower  than  the  slowest  estimates  for 
anarchic  deployments. 

General  Discussion:  Implications  and  Applications 

The  evidence  from  our  experiments  indicates  that  memory  for  previously  attended  items 

i<  ef-  .... 

jf  limited  if  any  use  in  guiding  subsequent  deployments.  Note  that  this  conclusion  is 

confined  to  the  rapid,  covert  deployments  of  attention  made  in  visual  searches  of  about 

one  second  duration.  There  is  no  doubt  that  a  slower,  strategic  memory  has  an  impact  on 

more  extended  searches.  If  you  search  your  kitchen  for  your  keys  and  move  on  to  the 

living  room,  you  know  that  you  have  searched  the  kitchen  and  that  knowledge  inhibits 

further  searching  of  the  kitchen.  Of  course,  you  may  return  to  look  again  but  that  is 

behavior  on  a  completely  different  time  scale  than  what  is  under  discussion  here. 

Why  not  let  memory  for  rejected  distractors  guide  attention?  The  set  of  experiments  on 
commanded  deployments  of  attention  provide  a  hint  but  not  an  answer.  Moving  attention 
on  a  specific  path  is  a  form  of  memory.  This  turns  out  to  slow  deployment  by 
substantially  more  than  a  factor  of  two.  Since  the  benefit  of  full  memory  over  sampling 
with  replacement  is  only  a  factor  of  two,  the  visual  system  would  seem  to  have  concluded 
that  this  solution  is  not  worth  the  time. 

The  slow  speed  of  planned  deployments  does  not  rule  out  all  memory  mechanisms. 
However,  the  basic  explanation  may  generalize.  Memory  may  not  be  worth  the  costs. 
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One  of  the  aims  of  this  grant  is  to  continue  work  on  the  Guided  Search  (GS)  model.  Our 
current  work  on  Guided  Search  4  is  a  logical  outgrowth  of  work  on  Guided  Search 
versions  1  and  2  (Wolfe,  1994;  Wolfe  et  al.,  1989).  It  is  not  a  direct  descendent  of  GS3 
(Wolfe  &  Gancarz,  1996).  GS3  was  developed  prior  to  our  work  on  memory  for  rejected 
distractors.  It  was  intended  to  incorporate  eye  movements  into  the  model.  Most  real- 
world  searches  do  not  constrain  eye  movements  and  most  of  our  data  come  from 
experiments  in  which  the  eyes  are  free  to  move.  If  you  want  to  keep  track  of  rejected 
distractors  and  you  can  move  your  eyes,  then  the  rejected  distractors  cannot  be  marked  in 
simple  retinotopic  coordinates.  Most  of  the  work  in  GS3  was  involved  in  coordinate 
transformation,  much  of  it  to  provide  the  inhibition  of  rejected  distractors  that  we  then 
thought  was  needed.  The  model  is  simplified  if  that  requirement  is  removed. 

Of  course,  we  do  not  know  if  the  design  of  our  attentional  mechanisms  was  limited  by  the 
same  considerations  that  influence  our  efforts  to  model  those  mechanisms.  But  it  is 
certainly  plausible  that  using  the  entire  prior  history  of  a  search  to  govern  the  next  steps 
in  that  search  was  simply  too  “expensive”  and  not  worth  the  modest  gains  in  search 
efficiency.  Our  results  and  modeling  do  suggest  that  artificial  searchers  might  make  use 
of  the  same  “simple-minded”  (and  amnesic)  principles  that  human  search  seems  to 
employ. 

Our  efforts  to  put  together  a  comprehensive  GS4  model  have  been  slowed  by  the  fast 
pace  of  data  collection.  Each  time  the  modeling  gets  well  underway,  we  make  an 
empirical  discovery  that  requires  that  we  alter  the  model.  The  work  on  memory  in  search 
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seemed  to  raise  one  serious  problem.  How  could  a  system  with  no  memory  avoid 
perseveration.  Why  wouldn’t  such  a  system  deploy  its  attention  to  the  single  most  salient 
item  over  and  over  again?  We  have  examined  this  danger  by  building  versions  of  the 
Guided  Search  model  that  sample  with  replacement  (Wolfe,  2001).  The  change  from 
sampling  without  replacement  to  sampling  with  replacement  turns  out  not  to  be 
particularly  problematic.  Guided  Search  creates  an  “activation  map”  that  rank  orders 
items  and  deploys  attention  to  the  item  with  the  highest  activation.  For  items  that  are  all 
otherwise  equivalent  to  each  other,  perseveration  is  avoided  if  the  model  has  some 
dynamic  noise  that  changes  the  rank  ordering  of  items. 

The  core  of  Guided  Search  is  the  idea  that  basic  feature  information  can  guide  attention. 
Activation  of  some  items  is  higher  than  others  because  they  differ  from  their  neighbors  in 
a  “bottom-up”  manner  (Itti  &  Koch,  2000;  Li,  2002)  or  because  they  possess  features  that 
are  selected  as  target  features  in  a  top-down  manner  (Egeth,  1977;  Egeth,  Virzi,  & 
Garbart,  1984;  Wolfe,  Horowitz,  &  Kenner,  2003).  Guidance  is  not  altered  by  the  lack  of 
memory  for  rejected  distractors.  If  an  observer  is  looking  for  red  Ts  among  green  Ts  and 
red  Ls,  attention  will  be  deployed  at  random  among  the  red  items.  Even  sampling  with 
replacement,  the  model  will  not  sample  green  items  with  low  activation. 

Sampling  with  replacement  does  pose  a  number  of  challenges.  For  example,  how  would 
an  observer  know  when  to  stop  and  declare  the  target  absent?  Our  current  work  tackles 
such  challenges. 
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